Self-talk Discrimination in Human-Robot Interaction Situations for Supporting Social Awareness

Being aware of the presence, activities and is fundamental for Human–Robot Interaction and assistive applications. In this paper, we describe (1) designing triadic situations for cognitive stimulation for elderly users; (2) characterizing social signals that describe social context: system directed speech (SDS) and self-talk (ST); and (3) estimating an interaction efficiency measure that reveals the quality of interaction. The proposed triadic situation is formed by a user, a computer providing cognitive exercises and a robot that provides encouragement and help using verbal and non-verbal signals. The methodology followed to design this situation is presented. Wizard-of-Oz experiments have been performed and analyzed through eyecontact behaviors and dialog acts (SDS and ST). We show that users employ two interaction styles characterized by different prosody features. Automatic recognition systems of these dialog acts is proposed using k-NN, decision tree and SVM classifiers trained with pitch, energy and rhythmicbased features. The best recognition system achieves an accuracy of 71 %, showing that interaction styles can be discriminated on the basis of prosodic features. An Interaction Efficiency (IE) metric is proposed to characterize interaction styles. This metric exploits on-view/off-view discrimination, semantic analysis and ST/SDS discrimination. Experiments on collected data prove the effectiveness of the IE This work has been supported by French National Research Agency (ANR) through TecSan program (Robadom project ANR-09-TECS-012). J. Le Maitre · M. Chetouani ( ) ISIR UMR 7222, Université Pierre et Marie Curie, Paris, France e-mail: [email protected] J. Le Maitre e-mail: [email protected] measure in evaluating the individual’s quality of interaction of elderly patients during the cognitive stimulation task.